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Abstract 



This technical report provides a summary of feedback from teachers, administrators, and support 
personnel who used the easyCBM progress monitoring and benchmark assessment system during 
school year 2009/2010. Data were gathered from semi- structured focus groups conducted during 
the 2010 easyCBM August Institute at the University of Oregon. Results have been used to 
prioritize enhancements to the easyCBM system and identify additional areas of need for future 
development and training. 




easyCBM Focus Group 



Page 1 



Making the Good Even Better: Feedback from easyCBM Focus Groups, 

School Year 2009/2010 

The easyCBM progress monitoring and benchmark assessment system (Alonzo, Tindal, 
Ulmer, & Glasgow, 2006) was developed as part of a model demonstration project funded by the 
Office of Special Education Programs in 2006. Initially available only to a few school districts in 
the service area of the university where it was developed, easyCBM was first made available to a 
larger group of educators in the Fall of 2009. At the time this technical report was written, 24 
school districts in Oregon had adopted the District version of easyCBM, joined by 8 districts in 
Washington state, 1 district in Mississippi, 2 in Illinois, 2 in Pennsylvania, 1 in Indiana, 1 in 
Ohio, 1 in North Carolina, 1 in Kansas, and 1 in Maine. As these numbers suggest, the system 
has received attention from a national audience and use continues to grow exponentially. 

As part of the developers’ ongoing effort to ensure that the system meets educators’ 
needs, we conducted a series of studies in the spring and summer of 2010 to gather input from 
teachers and administrators with easyCBM accounts. This technical report provides information 
from the focus group studies. 

Methods 

To capture as much information from as diverse a user group as possible, we conducted 
semi-structured focus groups, involving experienced users of district easyCBM and novice users 
the system on two separate days. 

Setting and Subjects 

The focus groups were held as part of the 2010 easyCBM August Institute that took 
place at the University of Oregon. The focus groups were conducted in two parts, with two 
different participant populations. Participants in the August 9 th focus group included school and 
district administrators, and general and special education teachers representing five different 




easyCBM Focus Group 



Page 2 



districts that had used the district version of easyCBM in School Year 2009/2010. Participants in 
the August 10 th focus group included school and district administrators, general and special 
education teachers, and Educational Service District representatives from the state of Oregon, all 
of whom would be using the district version of easyCBM for the first time in the Fall of the 
2010/2011 school year. Approximately 80% of participants in the August 10 th focus group had 
participated as silent observers in the August 9 th focus group. 

Measurement/Instrument Development 

The semi- structured focus groups were conducted as part of the 2010 easyCBM August 
Institute hosted by Behavioral Research and Teaching at the University of Oregon. Participants 
were provided with the results of two surveys conducted earlier in the spring and summer and 
asked to respond to questions related to their schools’ and districts’ capacity to use the computer- 
based easyCBM system, the student information database they used in their districts and its 
ability to interface with the easyCBM system, the relation between use of the easyCBM system 
and student performance on the Oregon statewide assessment of reading and mathematics, and 
the adequacy of the training available to them. 

During the August 9 th focus group, attendees from districts that had used the district 
version of easyCBM during School Year 2009/2010 were asked to sit in a circle, facing one 
another. Attendees from districts preparing to adopt the system for initial use in School Year 
2010/2011 sat in a larger circle, surrounding the first group. They were asked to jot down notes 
on topics they thought were most relevant during the focus group. These notes were later 
transcribed by a research assistant who had attended the August easyCBM Institute. The 
facilitator, the first author on this technical report, posed questions to the group and encouraged 
participation from all attendees, with follow-up questions as they arose. The third author on this 
technical report served as an observer and note-taker during the focus group sessions. 
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For the August 10 th focus group, attendees were first provided approximately 4 hours’ 
training on the district version of easyCBM. Following a lunch break, they were then brought 
back together for the actual focus group. During their focus group time, attendees were asked to 
reflect on their observations from the prior day’s focus group as well as their notes from the 
training provided earlier that day, and to share questions and concerns related to their capacity to 
use the district easyCBM system. The same facilitator, note-takers and observers participated in 
the same roles on both days, providing consistency across the two groups. 

Data Preparation and Analysis 

Data from the survey were gathered through the online survey tool SurveyMonkey, 
which presents summaries of responses to each question as well as lists of all constructed- 
responses provided. These data are reported in full in this technical report. 

Data from the focus groups were compiled by the third author on this technical report and 
are reported in full in this technical report. 

Results 

The focus groups addressed areas of technology needs and capacity, training needs and 
capacity, and additional resource needs. Findings from each of these three broad categories are 
presented separately. 

Technology Needs and Capacity 

Focus group participants reported a variety of student information systems, with the 
following programs referenced: Pentamation, School Master, ESIS, and Skyward. All 
participants indicated that their student information system interfaced easily with the easyCBM 
system, allowing smooth and accurate transfer of student and staff data across the different 
platforms. Participants indicated strong interest in having the Oregon statewide assessment 
scores included in the easyCBM reports, indicating that this additional piece of data would 
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facilitate decision making. A few participants discussed a need for a more automated means of 
updating the student records on easyCBM, and there was some discussion about inter-district 
collaboration on writing the computer code that would facilitate this happening. 

Although participants on both days indicated that the demands for computer lab time to 
finish the state assessment in the spring presented challenges for finishing the spring benchmark 
assessments on easyCBM, they also indicated that this was not as much of a problem during the 
fall or winter benchmark windows. Participants’ benchmark windows ranged from a couple of 
days for the smallest districts to three weeks to complete the full set of benchmark assessments 
across their district. All participants on both days indicated that the data download features on 
easyCBM were accessible and easily understood. 

The biggest challenge, related to technology, identified by participants involved correctly 
linking students with adult staff using the student/teacher association table. This problem 
appeared to be a challenge primarily because this information did not necessarily already exist in 
the Student Information Systems used by the different districts. Returning district users on 
August 9 th mentioned the importance of using the unique State Student Identification number 
rather than a locally-created student ID number as the system expands so that students from 
different districts can be passed from one district to the next without losing any of their historical 
easyCBM data. 

Training Needs and Capacity 

Participants on both days were provided with the PowerPoint presentation prepared by 
the lead author on this technical report as a tool to introduce others to the district easyCBM site. 
Participants in the August 9 th focus group indicated that similar PowerPoint presentations that 
had been made available to them the prior year when they received their initial training on the 




easyCBM Focus Group 



Page 5 



system had been useful to them as they provided professional development to others in their 
districts. 

Some participants reported that they found the training videos and materials available on 
the easyCBM site useful as they worked with others in their districts. A common theme among 
participants on both days was the need for ongoing professional development, not only on using 
the easyCBM system itself but also on instructional interventions and strategies to support 
students whose easyCBM performance indicated they needed additional academic support. A 
common theme on both days was the need for more information related to mathematics. Some 
participants on both days referenced the Oregon Data Project and emphasized the connections 
they saw between that state effort and effective use of the easyCBM system. 

Participants expressed support for ongoing collaborative networking between districts 
using easyCBM in the state. They indicated that such collaboration could help provide 
‘shortcuts’ for new districts starting to use the system by providing a network of educators from 
across the state who might be able to assist with questions related to uploading student and staff 
data, interpreting the results, or recommending instructional programs, strategies, and 
interventions. In addition, participants indicated that involvement of Educational Service 
Districts (ESDs) throughout the state would help provide the infrastructure needed to expand use 
of the easyCBM system, particularly in the smaller and more remote districts. Representatives 
from three ESDs were present during both focus groups, and they added their support for this 
suggestion. 

Additional Resource Needs 

Participants of both focus groups shared their enthusiasm for developing a “resource and 
training” system where districts could place resources they had developed to assist in the use of 
benchmarking and progress monitoring data to guide instructional decision making. One 
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participant shared concerns about quality control, and others in attendance agreed that there 
would be a need for any resources to be ‘vetted’ prior to being placed on the site for 
dissemination to ensure that they met quality standards. During the August 10 th focus group, 
participants discussed the potential for using a more- widely fleshed out data system that also 
included specific information about students’ program of study and the specific instructional 
strategies, curriculum, and interventions they were being provided. They thought such a data 
system might provide much-needed data on the effectiveness of different approaches, 
particularly in mathematics, where little research currently exists. 

Discussion 

Results of the focus groups provide evidence that the current district easyCBM system is 
widely-accessible and provides useful information to districts in Oregon. The existing 
technology interfaces well with a variety of student information systems in use across the state 
and provides easily-accessible student data appropriate for district, building, and classroom-level 
educators. The existing training materials available on the easyCBM system, which provide 
instruction on administering and scoring the benchmarking and progress monitoring assessments 
(training link on the easyCBM home page) and on interpreting the benchmark, group, and 
individual reports (teachers’ manual, downloadable PDF on the easyCBM home page), meet 
district needs for those particular areas. However, participants clearly identified a need for 
additional information on how to intervene with students once they are identified as being at risk, 
particularly in the area of mathematics. 
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